A modern trend for using network connections for sending and receiving large volumes of packetized data over long distance (often between continents) presents some problems with regards to a high round trip time (RTT) and a high number of lost packets (P-lost). In other words, these networks have long delays and can have many undelivered (i.e., lost) packets. The conventional TCP mechanism does not work well in these situations—i.e., a size of the TCP window decreases automatically, resulting in a low number of transferred packets.
The TCP window determines maximum bandwidth of a connection channel. For example, if 1 MB of data can be transferred at one instance, but the latency (RTT) of the channel is 200 msec, only 5 MB of data can be transferred over 1 sec period, even if the physical channel bandwidth is 100 MB. In other words, a channel with a high bandwidth would still produce a very low data transfer rate in this case. Often, this occurs not due to physical limitations of the channel, but because of restrictions in how network providers manage bandwidth and divide it among multiple users—a problem that is particularly acute in long-distance data transmission. This issue is not adequately addressed by any existing network optimization solutions.
Several customized protocols are used in order to compensate for shortcomings of the TCP, for example, UDP-based Data Transfer protocol (UDT) and Reliable WAN Transfer Protocol (RWTP) by TIXEL Inc. These protocols essentially offer a TCP-like service, implementing customized similar ideas and methods, where the transfer window is regulated by other rules and can be much larger than typical TCP window, such as, for example, a 100 MB window. Unlike the conventional TCP, if the packet is lost, the size of the transfer window is not immediately affected. In the conventional TCP, if three packets are lost, the size of the TCP Window is reduced by 50% and then begins to increase and gradually reaches the initial size. Unlike the TCP, these alternative solutions attempt to increase the size of the transfer window or at least keep the same size of the window in case of losses of packets.
These approaches cause several problems. A server cannot service a large number of clients, because windows from each of the clients are accumulated on the server when the size of the TCP Window is increased. For example, in case of 1,000 clients, e.g., 100 GB may be required for connections only. In case of a larger number of clients, an enormous volume of memory may be needed. Another problem is limitations of the Network Address Translation (NAT) mechanism, which needs to be addressed. See, e.g., https://en.wikipedia.org/wiki/UDPholepunching. For example, UDP hole punching will not work with symmetric NAT devices (also known as bi-directional NAT) which tend to be found in large corporate networks, etc. However, the original TCP does not have these problems, because the load is evenly balanced among the sockets. Yet the standard TCP can be slow and inefficient in terms of RTT and P-lost.
Accordingly, an efficient method for optimization of standard TCP network connections by redistribution of loads at a system kernel level is desired.